The invention relates generally to range finder type image sensors, and more particularly to such sensors as may be implemented on a single integrated circuit using CMOS fabrication, and especially to reducing power consumption of systems utilizing such sensors.
Electronic circuits that provide a measure of distance from the circuit to an object are known in the art, and may be exemplified by system 10 FIG. 1. In the generalized system of FIG. 1, imaging circuitry within system 10 is used to approximate the distance (e.g., Z1, Z2, Z3) to an object 20, the top portion of which is shown more distant from system 10 than is the bottom portion. Typically system 10 will include a light source 30 whose light output is focused by a lens 40 and directed toward the object to be imaged, here object 20. Other prior art systems do not provide an active light source 30 and instead rely upon and indeed require ambient light reflected by the object of interest.
Various fractions of the light from source 30 may be reflected by surface portions of object 20, and is focused by a lens 50. This return light falls upon various detector devices 60, e.g., photodiodes or the like, in an array on an integrated circuit (IC) 70. Devices 60 produce a rendering of the luminosity of an object (e.g., 10) in the scene from which distance data is to be inferred. In some applications devices 60 might be charge coupled devices (CCDs) or even arrays of CMOS devices.
CCDs typically are configured in a so-called bucket-brigade whereby light-detected charge by a first CCD is serial-coupled to an adjacent CCD, whose output in turn is coupled to a third CCD, and so on. This bucket-brigade configuration precludes fabricating processing circuitry on the same IC containing the CCD array. Further, CCDs provide a serial readout as opposed to a random readout. For example, if a CCD range finder system were used in a digital zoom lens application, even though most of the relevant data would be provided by a few of the CCDs in the array, it would nonetheless be necessary to readout the entire array to gain access to the relevant data, a time consuming process. In still and some motion photography applications, CCD-based systems might still find utility.
As noted, the upper portion of object 20 is intentionally shown more distant that the lower portion, which is to say distance Z3 greater than Z3 greater than Z1. In a range finder autofocus camera environment, one might try to have devices 60 approximate average distance from the camera (e.g., from Z=0) to object 10 by examining relative luminosity data obtained from the object. In some applications, e.g., range finding binoculars, the field of view is sufficiently small such that all objects in focus will be at substantially the same distance. But in general, luminosity-based systems do not work well. For example, in FIG. 1, the upper portion of object 20 is shown darker than the lower portion, and presumably is more distant than the lower portion. But in the real world, the more distant portion of an object could instead be shinier or brighter (e.g., reflect more optical energy) than a closer but darker portion of an object. In a complicated scene, it can be very difficult to approximate the focal distance to an object or subject standing against a background using change in luminosity to distinguish the subject from the background. In such various applications, circuits 80, 90, 100 within system 10 in FIG. 1 would assist in this signal processing. As noted, if IC 70 includes CCDs 60, other processing circuitry such as 80, 90, 100 are formed off-chip.
Unfortunately, reflected luminosity data does not provide a truly accurate rendering of distance because the reflectivity of the object is unknown. Thus, a distant object surface with a shiny surface may reflect as much light (perhaps more) than a closer object surface with a dull finish.
Other focusing systems are known in the art. Infrared (IR) autofocus systems for use in cameras or binoculars produce a single distance value that is an average or a minimum distance to all targets within the field of view. Other camera autofocus systems often require mechanical focusing of the lens onto the subject to determine distance. At best these prior art focus systems can focus a lens onto a single object in a field of view, but cannot simultaneously measure distance for all objects in the field of view.
In general, a reproduction or approximation of original luminosity values in a scene permits the human visual system to understand what objects were present in the scene and to estimate their relative locations stereoscopically. For non-stereoscopic images such as those rendered on an ordinary television screen, the human brain assesses apparent size, distance and shape of objects using past experience. Specialized computer programs can approximate object distance under special conditions.
Stereoscopic images allow a human observer to more accurately judge the distance of an object. However it is challenging for a computer program to judge object distance from a stereoscopic image. Errors are often present, and the required signal processing requires specialized hardware and computation. Stereoscopic images are at best an indirect way to produce a three-dimensional image suitable for direct computer use.
Many applications require directly obtaining a three-dimensional rendering of a scene. But in practice it is difficult to accurately extract distance and velocity data along a viewing axis from luminosity measurements. Nonetheless many applications require accurate distance and velocity tracking, for example an assembly line welding robot that must determine the precise distance and speed of the object to be welded. The necessary distance measurements may be erroneous due to varying lighting conditions and other shortcomings noted above. Such applications would benefit from a system that could directly capture three-dimensional imagery.
Although specialized three dimensional imaging systems exist in the nuclear magnetic resonance and scanning laser tomography fields, such systems require substantial equipment expenditures. Further, these systems are obtrusive, and are dedicated to specific tasks, e.g., imaging internal body organs.
In other applications, scanning laser range finding systems raster scan an image by using mirrors to deflect a laser beam in the x-axis and perhaps the y-axis plane. The angle of defection of each mirror is used to determine the coordinate of an image pixel being sampled. Such systems require precision detection of the angle of each mirror to determine which pixel is currently being sampled. Understandably having to provide precision moving mechanical parts add bulk, complexity, and cost to such range finding system. Further, because these systems sample each pixel sequentially, the number of complete image frames that can be sampled per unit time is limited.
In summation, there is a need for a system that can produce direct three-dimensional imaging. Preferably such system should be implementable on a single IC that includes both detectors and circuitry to process detection signals. Such single IC system should be implementable using CMOS fabrication techniques, should require few discrete components and have no moving components. Optionally, the system should be able to output data from the detectors in a non-sequential or random fashion. Very preferably, such system should require relatively low peak light emitting power such that inexpensive light emitters may be employed.
The present invention provides such a system.
The present invention provides a system that measures distance and velocity data in real time using time-of-flight (TOF) data rather than relying upon luminosity data. The system is CMOS-compatible and provides such three-dimensional imaging without requiring moving parts. The system may be fabricated on a single IC containing both a two-dimensional array of CMOS-compatible pixel detectors that sense photon light energy, and associated processing circuitry.
In applicant""s referenced utility application, now U.S. Pat. No. 6,323,942 (2001), a microprocessor on the IC continuously triggered a preferably LED or laser light source whose light output pulses were at least partially reflected by points on the surface of the object to be imaged. For good image resolution, e.g., a cm or so, a large but brief pulse of optical energy was required, for example, a peak pulse energy of perhaps low, a pulse width of about 15 ns, and a repetition rate of about 3 Khz. While average energy in applicant""s earlier system was only about 1 mW, the desired 10 W peak power essentially dictated the use of relatively expensive laser diodes as a preferred energy light source. Each pixel detector in the detector array had associated electronics to measure time-of-flight from transmission of an optical energy pulse to detection of a return signal. In that invention, the transmission of high peak power narrow energy pulses required the use of high bandwidth pixel detector amplifiers.
By contrast, the present invention transmits periodic signals having a high frequency component, which signals have low average Power and low peak power, e.g., tens of mW rather than watts. Periodic signals such as an ideal sinusoid S1=cos(xcfx89xc2x7t) having of optical energy are relatively straightforward to analyze and will be assumed herein. Emitting low peak power periodic signals with a high frequency component such as sinusoidal optical signals permits using inexpensive light sources and simpler, narrower bandwidth pixel detectors. Bandwidths can be on the order of a few hundred KHz with an operating (emitted energy) frequency of about 200 MHz. Good resolution accuracy is still obtainable using a low peak power optical emitter in that the effective duty cycle is greater than the output from a narrow-pulsed optical emitter of higher peak power.
Assume that the energy emitted from the optical source is approximately S1=Kxc2x7cos(xcfx89xc2x7t) where K is an amplitude coefficient, xcfx89=2xcfx80f, and frequency f is perhaps 200 MHz, that distance z separates the optical energy emitter from the target object. For ease of mathematical representation, K=1 will be assumed although coefficients less than or greater than one may be used. The term xe2x80x9capproximatelyxe2x80x9d is used in recognition that perfect sinusoid waveforms can be difficult to generate. Due to the time-of-flight required for the energy to traverse distance z, there will be a phase shift "PHgr" between the transmitted energy and the energy detected by a photo detector in the array, S2=Axc2x7cos(xcfx89xc2x7+"PHgr"). Coefficient A represents brightness of the detected reflected signal and may be measured separately using the same return signal that is received by the pixel detector.
The phase shift "PHgr" due to time-of-flight is:
"PHgr"=2xc2x7xcfx89xc2x7z/C=2xc2x7(2xc2x7xcfx80xc2x7f)xc2x7z/C 
where C is speed of light 300 Km/sec. Thus, distance z from energy emitter (and from detector array) is given by:
z="PHgr"xc2x7C/2xc2x7xcfx89="PHgr"xc2x7C/{2xc2x7(2xc2x7xcfx80xc2x7f)}
Distance z is known modulo 2xcfx80C/(2xc2x7xcfx89)=C/(2xc2x7f). If desired, several different modulation frequencies of optically emitted energy may be used, e.g., f1, f2, f3 . . . , to determine z modulo C/(2xc2x7f1), C/(2xc2x7f2), C/(2xc2x7f3). The use of multiple different modulation frequencies advantageously can reduce aliasing. If f1, f2, f3 are integers, aliasing is reduced to the least common multiplier of f1, f2, f3, denoted LCM(f1, f2, f3). If f1, f2, f3 are not integers, they preferably are modeled as fractions expressible as a1/D, a2/D, and a3/D, where i in ai is an integer, and D=(GCD) represents the greatest common divisor of a1, a2, a3. From the above, distance z may be determined modulo LCM(a1, a2, a3)/D.
In the present invention, phase "PHgr" and distance z are determined by mixing (or homodyning) the signal detected by each pixel detector S2=Axc2x7cos(xcfx89xc2x7t+"PHgr") with the signal driving the optical energy emitter S1=cos(xcfx89xc2x7t). The mixing product S1xc2x7S2 will be 0.5xc2x7Axc2x7{cos(xcfx89xc2x7t+"PHgr")+cos("PHgr")} and will have an average value of 0.5xc2x7Axc2x7cos("PHgr"). If desired, the amplitude or brightness A of the detected return signal may be measured separately from each pixel detector output.
To implement homodyne determination of phase "PHgr" and distance z, each pixel detector in the detector array has its own dedicated electronics that includes a low noise amplifier to amplify the signal detected by the associated pixel detector, a variable phase delay unit, a mixer, a lowpass filter, and an integrator. The mixer mixes the output of low noise amplifier with a variable phase delay version of the transmitted sinusoidal signal. The mixer output is lowpass filtered, integrated and fedback to control phase shift of the variable phase delay unit. As such, the phase signal "PHgr" is obtained by homodyning the received signal S2 with a phase-delayed version of the emitted signal S1 whose phase is dynamically forced to match the phase of S2 by closed-loop feedback. As noted above, phase shift "PHgr" as well as amplitude or brightness information A are obtained. In the equilibrium state, the output of each integrator is the phase "psgr" (where "psgr"="PHgr"xc2x1xcfx80/2) associated with the TOF or distance z between the associated pixel detector and a point a distance z away on the target object. The analog phase information is readily digitized, and an on-chip microprocessor can then calculate z-values from each pixel detector to an associated point on the target object. The microprocessor further can calculate dz/dt (and/or dx/dt, dy/dt) and other information if desired.
The on-chip measurement information may be output in random rather than sequential order, and object tracking and other measurements requiring a three-dimensional image are readily made. The overall system is small, robust and requires relatively few off-chip discrete components. On-chip circuitry can use such TOF data to readily simultaneously measure distance and velocity of all points on an object or all objects in a scene.
Other features and advantages of the invention will appear from the following description in which the preferred embodiments have been set forth in detail, in conjunction with their accompanying drawings.